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ABSTRACT 


The military relies heavily on computer systems. Without a strong method of authentication 
to access these systems, threats to confidentiality, integrity, and availability of government 
information are likely to be more successful. A recent method of authentication for the 
Windows 8 and Windows 10 operating systems is picture gesture authentication (PGA), 
a new approach to entering a password to authenticate a user during system login. Each 
PGA password is composed of three gestures that are drawn over a picture chosen by the 
user. Strength requirements are set for PGA passwords similarly to text-based passwords. 
For simplicity, users tend to use shapes, colors, and objects in a picture, called points of 
interest (POI), as guidance when creating each gesture for their password. This concept 
provides an opportunity for potential hackers to make logical password guesses, decreas¬ 
ing the security of PGA. Previous work on PGA security used a proprietary brute-force 
algorithm to guess passwords based on POIs. We present a similar brute-force algorithm 
that is publicly available. We evaluate the efficiency of the new algorithm against various 
background pictures and propose strength requirements to improve the security of PGA. 
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CHAPTER 1: 

Introduction 


1.1 Motivation 

The use of passwords as a method of authenticating someone’s claim of identity dates back 
to ancient times in the Roman military in which Romans referred to passwords as “watch¬ 
words” [1], Since then, passwords have been modified to what we have today. Traditional 
computer-based authentication methods use text-based passwords, which are a string of 
alphanumeric characters and symbols used to authenticate a user before granting that user 
access to a device or program. For security purposes, many programs use strength require¬ 
ments for passwords. Strength requirements may include a certain number of uppercase 
alphabetic characters, lowercase alphabetic characters, symbolic characters, or numerical 
characters, and a minimum and maximum length. They may also require a password to be 
changed after a period of time, and that no repeated passwords may be used. Even with 
these strength requirements, there remain weaknesses in text-based passwords. 

Suo et al. said “human factors are often considered the weakest link in a computer security 
system” [2], Zhao et al. found that people use simple passwords because they are easier 
to remember [3], [4], Therefore, dictionary attacks were created, where a list of plausible 
passwords are generated based on dictionary words, and used to guess passwords. Other 
human factors related to text-based passwords include users recycling passwords through¬ 
out different programs or re-using passwords for the same program. Users also tend to 
write down their password, either on a sticky note left on their desk or in an unencrypted 
document on their system. In either case, if the password is found there can be numerous 
consequences to security. If a password gets in the wrong hands, it can lead to illicit access 
onto a private network, or a data breach. 

Since text-based passwords are difficult for people to keep track of, other methods of au¬ 
thentication have been developed. Suo et al. believe that people are more likely to remember 
a visual password [2], Picture gesture authentication (PGA) is a new type of authentication 
that uses picture-based passwords, and is the scope of this thesis. In particular, the research 
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looks at the best types of background pictures for more secure PGA. 

We proceed by presenting a brute-force algorithm, designed after the work of Zhao et 
al. [3], [4], that makes logical guesses to crack the PGA password of a user given a specific 
picture. We programmed the algorithm to use points of interest (POI), which are specific 
areas of a picture that may catch the eye of a user, to determine likely choices of a pass¬ 
word. By analyzing the accuracy and efficiency of the algorithm to generate brute-force 
passwords, we determine the variety of pictures that are superior for a background picture. 
We show that the background picture selected can increase the strength of the password 
chosen for PGA. 


1.2 Benefits to the Navy 

The main contribution of this research is to investigate the security bounds of picture ges¬ 
ture authentication. The Navy would benefit from this study because if PGA is not a strong 
method of authentication, then potential threats to confidentiality, integrity, and availability 
of government information are plausible. Strong authentication is recommended by the 
DOD Cybersecurity Discipline Implementation Plan that was amended February 2016. 


Reducing anonymity as well as enforcing authenticity and accountability for 
actions on DOD information networks improves the security posture of the 
DOD. The connection between weak authentication and account takeover is 
well-established. Strong authentication helps prevent unauthorized access, in¬ 
cluding wide-scale network compromise by impersonating privileged adminis¬ 
trators. Commanders and Supervisors will focus attention on protecting high- 
value assets, such as servers and routers, and privileged system administrator 
access. This line of effort supports objective 3-4 in the DOD Cyber Strat¬ 
egy, requiring the DOD CIO to mitigate known vulnerabilities by the end of 
2016. [6] 


An agreement between Microsoft and the DOD provides the Navy with the newest versions 
of Microsoft products, including Windows 8 and 10, which both use PGA. Navy Rear Ad¬ 
miral David G. Simpson, DISA’s vice director and senior procurement executive explained 
that the DOD has continued to focus on mobile computing, stating “Microsoft is committed 
to making sure that the technology within the agreement has a mobile-first focus, and we 
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expect to begin to take advantage of Microsoft’s mobile offerings as part of our enterprise 
mobility ecosystem” [2], Microsoft claims that PGA passwords are more secure than text- 
based passwords [5], and that DOD users will be more likely to use PGA. It is important, 
however, that PGA not be used in an insecure fashion, therefore, this study is important to 
help the Navy make the best decision on background pictures for the security of PGA. 


1.3 Thesis Organization 

The remainder of this thesis is organized as follows. In Chapter 2, we describe the history, 
notation, and brute-forcing of PGA, as well as related work. Chapter 3 discusses the two 
corpora used to test the program created for this thesis. The process of POI extractions and 
functions used for the brute-force algorithm are covered in Chapter 4. In Chapter 5, the 
passwords, POIs, and results of the program for each picture are explained. Finally, the 
conclusions and recommendations, and suggested future work of this thesis are presented 
in Chapter 6. 
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CHAPTER 2: 

Background and Related Work 


In this chapter, we explain the process of using picture gesture authentication (PGA) on 
a Windows 8 device. A key insight is how users tend to select points of interest (POI) to 
choose the location of gestures. POIs are a key concept employed by prior work on brute¬ 
forcing a password under PGA. We also summarize related work on picture authentication 
schemes. For clarity and ease of comparison, we adopt the notation of Zhao et al. [3], [4]. 

2.1 Picture Gesture Authentication 

Authentication is any mechanism used to validate if someone is the identity they claim to 
be on a computer system or program. There are three broad approaches to authentication, 
often referred to as something you know, something you own, or something you are. PGA 
is a relatively new authentication mechanism that falls under the umbrella of something 
you know. Microsoft started using PGA as an optional replacement for text passwords with 
their Windows 8 consumer technology. This new method of authentication was announced 
by Microsoft in late 2011 [5] for all versions of Windows 8 and products supporting PGA 
as a primary method of authentication were released on October 26, 2012. 

With the Windows 8 operating system, by default, user accounts are configured to use text- 
based passwords. To use PGA, the user selects the picture password sign-in option. After 
providing proper credentials, the user is required to choose a picture from their picture 
library. Using their own picture, instead of Microsoft providing one, will increase the 
security of PGA. The intuition is that two users are likely to select different pictures, as 
PGA is configurable per-user. After a picture is chosen, the user is prompted to create 
a password. A password for picture gesture authentication (PGA) is a series of gestures, 
limited to taps, circles or lines drawn on the picture. The users are expected to draw three 
gestures on the picture using their finger or stylus on the touchscreen or a mouse if no 
touchscreen is available. When users later authenticate with PGA, they must redraw the 
selected gestures, in the original order, on their chosen picture. 

We record a gesture password as a sequence of three gestures, n = ninjn?,. Each n is 
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one of many passwords in the password space, n e ]”[• Each gesture in the password is 
represented as a 7-tuple: 7r ( - = (g,xi,yi,X 2 ,y 2 ,f,d). Let g e {tap, circle, line} bo the 
type of gesture. The first coordinate (x \, y \) can indicate a tap point, the center of a circle 
or the first point of a line. The second coordinate ( X 2 , i/ 2 ) represents the end of a line, 
and is unused for other gesture types (i.e., let (x 2 , i/ 2 ) = (0,0) for a tap or circle). Let 
r be the radius of a circle gesture, and otherwise unused (i.e., r = 0 for a line or tap). 
Let d e {+, -,0} be the direction in which a circle is drawn, indicating a clockwise or 
counterclockwise gesture, and otherwise unused (i.e., 0 for a tap or line). Each gesture is 
one of many possible gestures in the gesture space, 7 r, e ["[• 


Figure 2.1 shows an example gesture password. The first gesture, n\ = (circle, 35,15,0,0,9, -), 
is a counterclockwise circle around the man’s head centered at (35,15) with a radius of 9. 

The second gesture, 712 = (line, 54,34,79,27,0,0), is a line from (54,34) to (79,27), from 
one woman’s nose to another’s. The last gesture, 773 = (tap, 16,35,0,0,0,0), is a tap on 
the left woman’s nose, at coordinate point (16,35). 



Figure 2.1. Example of a Sequence of Gestures on a Picture. Adapted 
from [3], [4], 


Naturally, human error is likely to occur when redrawing passwords. Therefore, a distance 
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function is built into the authentication process. Since pictures come in various sizes, the 
longest dimension is divided into 100 and the shortest dimension is scaled accordingly [5]. 
The pictures are scaled to determine the coordinate points that fall within an error distance 
of the actual coordinate point used for a gesture. When entering a password, if a coordinate 
point of a gesture is within the error distance of the actual coordinate point, that point will 
be accepted. 

Figure 2.2 shows an example of the points accepted during authentication within a distance 
of 3 around the recorded point of the actual password [5]. All of the gesture points within 
3 of the actual gesture point, shaded in green, are at least 90% accurate to the actual point 
within the error distance, and would be accepted during user login. The yellow, orange, and 
red points are not close enough to the actual gesture point to be accepted during user login. 
For example, a tap on (14,35) would suffice for the gesture n 3 = (tap, 16,35,0,0,0,0) 
since the distance d((l6, 35), (14, 35)) < 2V3. 
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Figure 2.2. Points < 90% to the 100% Exact Matched Point Are Accepted 
During Authentication. Adapted from [5]. 
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2.2 Brute-Forcing PGA 

Zhao et al. [3], [4] provide the intuition that users select gestures by employing points 
of interest (POIs) embedded in the underlying picture. POIs can be described by many 
features, such as objects Q> 0 = {head, eye,mouth,nose,bike, dog ,colors QJ C 
\ hi nr. red, yellow, green ,..shapes Q) s = {square, circle, triangle, rectangle ,...} 
and other miscellaneous types, £Z*. These form an attribute space D c2® such that = 
%U®cU®sU®*- Each POI is recorded as a 5-tuple, % = (xi, yi,X 2 , y 2 , D ), which 
defines the POI in picture k that is enclosed by a rectangle bounded by coordinates (xi, y\ ) 
and (x 2 , yi), and has the set of attributes r S. 

Referring back to the working example in Figure 2.1, the POIs include the heads of each 
person, their eyes, their noses, their mouths, the linear edge of the curtain, the blue lines 
in the man’s shirt, the black dots on the girl’s shirt, the woman’s necklace, and the corner 
of the vanity. Just as users tend to select dictionary words for text passwords, it is believed 
that they tend toward POIs on a picture to choose their PGA passwords. 

POIs help a user remember where they placed their gestures. This insight is used by Zhao 
et al. to provide an attack on PGA, comparable to a dictionary attack against text-based 
passwords. As mentioned in Section 2.1, it is unlikely that any two users would use the 
same picture for authentication. The attack framework requires previously seen passwords 
on known pictures to learn password-selection patterns to create a dictionary of gesture 
passwords. Machine analysis can then be used to identify POIs on pictures as a "dictionary" 
to guess a PGA password. This process is discussed in more detail in Chapter 3. 


2.3 Related Work 

There has been growing interest in providing an alternative to text passwords by using 
graphics. It has been argued that graphical passwords are more secure than text passwords, 
however, in “Graphical Passwords: A Survey,” Suo et al. explain how brute-force attacks, 
dictionary attacks, guessing, spyware, shoulder surfing, and social engineering are used 
to attack graphical passwords, just like text passwords [2]. They claim the defense against 
graphical passwords is more difficult since N length text passwords have 94 N possible pass¬ 
words based on 94 printable characters. On the other hand, PGA has only 1,155,509,083 
possible passwords with three gestures, based on all the possible sets of three gestures made 
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by taps, circles, and lines [5] whish is less than 94 6 , (the number of 6-character passwords). 
After guessing a graphical password, a program must be written to precisely draw such ges¬ 
tures on a picture. Suo et al. claimed in 2005 that there was no method of dictionary attacks 
on graphical passwords. Since then, research has shown that dictionary attacks are possible 
but must be designed for each individual picture, as described by Zhao et al. In Chapter 4, 
we explain how it is easy to guess graphical passwords since they are more predictable 
than text passwords. In 2013, Damopoulos et al. proved that there exists a touchlogger, 
similar to a keylogger but for touch screens, that can record gestures on touch screen de¬ 
vices [7], This is a finding made after Suo et al. stated that spyware was unable to track 
picture passwords. This is important to keep in mind since PGA is often used on, though is 
not limited to, touch screen devices. Picture passwords are vulnerable to shoulder surfing 
as we will discuss more in this section. Picture passwords are said to be insusceptible to 
social engineering because it is difficult to explain to someone verbally how to recreate a 
password [2]. 

One of the vulnerabilities of text passwords is that users tend to recycle passwords for 
separate accounts because it is difficult to remember multiple strong passwords. Suo et al., 
however, affirm that there is no “convincing evidence” that picture passwords are easier 
than text passwords to memorize. De Luca et al. also conclude that authentication methods 
other than text-based passwords and personal identification numbers (PIN) should be used, 
after analyzing password pattern authentication [8]. Pattern passwords consist of a series 
of continuous edges made on a 3x3 grid of points. They surveyed users over a period of 
time to collect data and study the passwords the users created, along with how they created 
them. Each user, they concluded, has a unique way of making each stroke. If used correctly, 
this pattern matching can be an additional method of authentication. Assuming an attacker 
knows the shape of the password, they may not be able to imitate the user’s stroke motions, 
which falls under the something you are category of authentication. 

Draw a Secret (DAS) is a picture-based password authentication method that allows a user 
to make a drawing on a blank grid as a password. This is different than PGA since there is 
not a background picture with POIs to guide users in creating a password, but is similar in 
the sense that a PGA password can be a picture drawn on a grid comprised of three gesture 
elements. Nali and Thorpe prove this scheme is insecure by showing that users center their 
drawings and use symmetry [9], [10], Essentially, they show that this approach increases 
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the chances of guessing a password. Dunphy and Yan attempted to enhance this method 
of authentication by providing a background picture for a user as a guide to improve how 
they originally created passwords [11]. This scheme is called Background Draw a Secret 
(BDAS). This relates to PGA since they both have a background picture that directs users 
in constructing a password, but BDAS has less restrictions on the number and types of 
gestures used for creating a password. They found that BDAS closely relates to PGA since 
users are likely to use POIs. Since BDAS is like PGA, and PGA is insecure, therefore 
BDAS is insecure. 

PassPoints is an authentication method that allows a user to choose points on a picture as 
a password. This is essentially a subset of the password space of PGA, with only the tap 
gesture being allowed. PassPoints is similar to DAS, containing a less structured pass¬ 
word space to PGA, but when selecting passwords PGA has fewer rules than PassPoints. 
Wiedenbeck et al. studied the security of PassPoints and found that users tend to use taps 
corresponding to POIs, which they call “hotspots,” when choosing points that correspond 
to POIs. The main outcome of their work is the recommendation that users should select 
pictures that avoid hotspots [12], [13]. 

Wiedenbeck et al. also found that users rely on POIs to assist in building passwords. Using 
the same dataset as Zhao et al. shown in Section 3.1, Alshehri et al. explored security of 
PGA, restricted to pictures with a high number of POIs. Since POIs are used to brute-force 
PGA, a background picture with more POIs would represent a larger password space, and 
thus provide more security against brute-force. As yet unpublished, they are developing 
a metric to find if a picture is suitably complex by validating pictures with more POIs to 
be less resistant to dictionary attacks [14]. Pictures with few POIs are more susceptible to 
attacks. Thus, Alshehri et al. claim there should be strength requirements of the background 
picture. In contrast, we are concerned with revalidating the premises and results of the 
original study by Zhao et al. 

Most PGA methods are used with touch screen devices. In addition to click points, as 
mentioned by Alshehri et al., Aviv et al. found that smudge marks can be used to guess the 
passwords of any of the aforementioned types of picture authentication [15], 

Picture password mechanisms are also susceptible to shoulder surfing. Logging in with 
PGA allows someone close by to easily see a user’s password. To provide more security, 
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a system such as LatentGesture can help keep the PGA password more secure [16]. La- 
tentGesture records a user’s behavior on a touchscreen device such as the speed of swiping 
across the screen or typing patterns. These recorded behaviours build a model of that user. 
When it suspects the current user does not match the model, LatentGesture will automat¬ 
ically log off the system. Saravana described a study using 20 people that were asked to 
check boxes, swipe sliding bars, and tap buttons to fill out a form. With high accuracy, 
LatentGesture was able to identify the users correctly [17]. This is not a surprising result 
because LatentGesture combines the something you know authentication category with the 
something you are category. 

Overall, picture gesture authentication has its weaknesses and vulnerabilities just like text- 
based passwords. Thus, we created a brute-force algorithm described in Chapter 4 to com¬ 
pare the security of one picture to another, determining the best selection of background 
pictures for an increase in security for PGA. Before describing the algorithm, we will dis¬ 
cuss the data given by Zhao et al. that we have also used in our study. 
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CHAPTER 3: 
Corpora 


In this chapter, we discuss the data gathered and analyzed by Zhao et al. in their study [4], 
Two corpora were employed in their study, both containing pictures and passwords created 
by the study’s subjects. The Arizona-Turk dataset was an artificial dataset, where subjects 
generated passwords for a small set of images. The Arizona-Student dataset was a more 
authentic dataset, where university students generated personal passwords used by a web¬ 
site. The next two sections summarize the demographics of the subjects and the contents 
of the corpora in the study. 


3.1 Arizona-T\irk Dataset 

The Arizona-Turk dataset (called dataset 2 in the Zhao et al. study [3], [4]) was solicited by 
advertisements in the schools of engineering and business at two different universities, and 
gathered using Amazon’s Mechanical Turk crowdsourcing service. Only individuals with 
previous security-related research experience were qualified to participate so they could 
understand the importance of this study. 

In the Arizona-Turk dataset, 762 subjects were given 15 pictures (see Figure 3.1) drawn 
from the PASCAL Visual Object Classes Challenge 2007 dataset [18]. The subjects were 
prompted to pretend the pictures were protecting their bank information, with the intention 
of influencing subjects to make strong passwords for each of the 15 pictures. Not all sub¬ 
jects completed the entire task, so the number of passwords gathered for each picture is 
not the same (see Figure 3.2). A total of 10,039 passwords were gathered: on average, 669 
passwords per picture and 13 passwords per subject. Interestingly, there were passwords 
which one might guess, such as circling tires on a bike and tapping a person’s nose. Further 
discussion can be found in Chapter 4. 

The subjects were given a demographic survey. Of the 762 subjects, only 652 (85.5%) 
filled out the survey. Of the 652 surveyed, 420 (64.4%) of them reported being male, 232 
(35.6%) female; 243 (37.2%) were between 18 and 24 years of age, 296 (45.4%) between 
25 and 34 years of age, and 98 (15%) between 35 and 50 years of age. 
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(a) 000243.jpg (b) 000316.jpg (c) 001116.jpg (d) 001358.jpg (e) 002057.jpg 



(f) 002080.jpg (g) 002840.jpg (h) 003026.jpg (i) 003731.jpg (j) 004054.jpg 



(k) 005570.jpg (I) 006412.jpg (m) 006467.jpg (n) 007628.jpg (o) 009899.jpg 

Figure 3.1. The 15 Pictures from the Arizona-Turk Dataset. Source: [3], [4], 


Number of Passwords Per Picture 



006412 007628 002080 001358 000243 003026 000316 002057 009899 001116 003731 004054 005570 002840 006467 

Picture 


Figure 3.2. Number of Passwords for Each of the 15 Pictures in the Arizona- 
Turk Dataset 
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As part of the survey, a multiple choice question was asked to help understand the choice 
of passwords made by the subjects as follows: “Which of the following best describes 
what you are considering when you choose locations to perform gestures?” Of the subjects 
in this study, 389 (59.6%) answered, “I try to find locations where special objects are”; 
143 (21.9%) answered, “I try to find locations where some special shapes are”; 57 (8.7%) 
answered, “I try to find locations where colors are different from their surroundings”; and 
66 (10.1%) answered, “I randomly choose a location to draw without thinking about the 
background picture.” Thus, 90.2% of respondents admitted to using a strategy of selecting 
POIs, which effectively limited the password space and, perhaps, biased it toward a POI 
populated area of the picture. 

3.2 Arizona-Student Dataset 

The Arizona-Student dataset (called dataset 1 in the Zhao et al. study) was gathered from 
university students in a classroom setting. An authentication method modeled after PGA 
in Windows 8 was created to gather information on how students in an undergraduate com¬ 
puter science class would create passwords. This authentication method was used by the 
students to access the course website, containing class materials such as homework, assign¬ 
ments, grades, and lecture notes. Data was gathered over one semester, or approximately 
three and a half months. 

The publicly released dataset contains subject IDs, a hash value for the picture, a password, 
and an activity log. The log recorded setting of passwords, attempted logins, the number 
of successful login attempts, and any password changes or new picture selections. Since 
students selected their own pictures, some contained family photos and other personally 
identifiable information (PII), so no pictures were released with the dataset. 

A total of 56 students in the computer science class participated in the study. The data col¬ 
lected reflected: 69 different pictures, 1 86 unique passwords, 2,536 login attempts (2,109 
successful, 427 failed) and 172 registrations (86 registered, 86 confirmations). On average, 
each student used 2.5 pictures, made 37.66 successful login attempts, had 7.625 failed login 
attempts, registered 1.53 logins, and confirmed 1.52 logins (see Figure 3.3). Between the 

According to Zhao et al. [3], [4], there were 58 unique pictures; this does not match the calculations 
made with the public-released data. 
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registrations, confirmations, and successful and failed logins, there were a total of 2,708 
datapoints. 


Successful and Failed Login Attempts per Subject 



Figure 3.3. Number of Successful/Failed Login Attempts and Number of 
Reset Passwords per Subject in the Arizona-Student Dataset 

The students were also asked the same demographic survey. Of the 56 students, only 33 
(58.9%) filled out the survey. Of the 33 surveyed, 27 (81.8%) reported being male, and 6 
(18.2%) female; 21 (63.6%) were between 18 and 24 years of age. Since the students were 
in an undergraduate course in computer science, it is reasonable that the numbers were not 
as diverse as those for dataset 2. 

As in the Arizona-Turk dataset, the subjects of the Arizona-Student dataset were also asked 
the question, “Which of the following best describes what you are considering when you 
choose locations to perform gestures?” Of the 33 respondents, 24 (72.7%) answered, “I try 
to find locations where special objects are”; 8 (24.2%) answered, “I try to find locations 
where some special shapes are”; 0 (0%) answered, “I try to find locations where colors are 
different from their surroundings”; and 1 (3%) answered, “I randomly choose a location 
to draw without thinking about the background picture.” Since students were asked to use 
this password to protect their actual course material, and to select their own pictures, we 
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expect that this dataset was more realistic than the Arizona-Turk dataset. This reflects an 
even stronger trend toward biased password selection. 
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CHAPTER 4: 
BestCover Algorithm 


This chapter presents the BestCover algorithm described by Zhao et al. [4] to make a 
logical guess of an unknown password using a previously unseen picture. In Section 4.1, we 
detail the POIs in the Arizona-Student dataset and the Arizona-Turk dataset. In Section, 4.2 
we define location dependent gesture selection functions and how they are used to map 
POIs into potential passwords for a picture. In Section 4.3, we then explain the BestCover 
algorithm, which uses a subset of the dataset for training and is evaluated on the remainder 
of the dataset. This is the same methodology employed by Zhao et al. to evaluate this 
algorithm [3], [4], We adopt the notation of Zhao et al. for ease of comparison between our 
independent re-implementation and their original work. 

4.1 POI Extraction 

For each of the datasets in the Arizona case study, Zhao et al. extracted POIs with “mature 
computer vision techniques such as object detection, feature detection and objectness mea¬ 
sure” [3], [4], The POI attributes were categorized as follows: face, body, eye, ear, mouth, 
nose, head/shoulder, clock, airplane, unknown object, forehead, car, line type, circle type, 
color type, “no semantics” and “not valid.” 

The number of POIs extracted from the pictures in the Arizona-Student dataset are ex¬ 
pressed in Figure 4.1. The number of POIs per picture varied widely between the pictures 
the students chose. Recall that for this dataset, some pictures were not made available due 
to PII concerns, however Zhao et al. [3], [4] provided information about the POIs (their 
type and their coordinate location on the picture). This eliminated the need to extract POIs 
using computer vision methods, and thus reduced many variables in the attempt to recreate 
an algorithm simliar to that of Zhao et al. to decide which background pictures are best to 
use in PGA. For the Arizona-Turk dataset, Figure 4.2 shows the number of POIs extracted 
for each of the 15 pictures in Figure 3.1. We observed a correlation between the variation 
in the number of POIs per picture, and the level of difficulty to brute force PGA passwords, 
described further in Chapter 5. 
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Number of POIs Per Picture 



Picture 


Figure 4.1. Number of POIs Extracted from the 58 Pictures in the Arizona- 
Student Dataset 

4.2 Location Dependent Gesture Selection Functions 

Users are likely to choose POIs on a picture when selecting a password. Therefore, map¬ 
pings were created to aid the brute force method described in Section 4.3. Location depen¬ 
dent gesture selection functions (LdGSF) [3], [4] are mappings s : Gx 2 ^ x 2 ^ x 0 —> 2 ^ 
from descriptions of gestures on POIs to PGA passwords using actual coordinate points of 
those POIs in the picture. The domain is the cross product of the set of gestures, the set of 
attributes at the first point, the set of attributes at the second point if the gesture is a line, 
and the set of POIs in the given picture, respectively. The range is the password space. 
Using the POIs extracted from the picture, as described in Section 4.1, a mapping can be 
made to describe gestures on a picture. A sequence of three LdGSF mappings, T = .V 1 .v 2 .v 3 , 
will yield three gestures, making plausible passwords. 

For example, referring to Figure 2.1 with the password n = n \ 7^3 , where n\ = 
(circle, 35,15,0,0,9, -), 712 = (line, 54, 34,79,27,0,0), and^ = (tap, 16,35,0,0,0,0), 
the LdGSFs for the k th picture pk would be: si = s(circle, {head},®,6k), £2 
slime, {nose}, {nose}, 6 k), S 3 = s(tap, { nose }, 0 , 6 k). 
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Number of POIs Per Picture: Method 2.1 



Picture 

Figure 4.2. Number of POIs Extracted from the 15 Pictures in the Arizona- 
Turk Dataset 

An LdGSF sequence can map to several passwords. For example, given the gesture made 
by .s i = sicircle, {head}, 0,6k) above, if a user decides to perform this gesture on Fig¬ 
ure 2.1, there are four possible heads to circle and each circle can have a different circum¬ 
ference. Therefore, one LdGSF can produce many possible gestures, and a single LdGSF 
sequence can produce many possible passwords. 


4.3 Brute-Force Algorithm 

Since POIs on a picture may decrease a user’s password space by steering them toward spe¬ 
cific gestures, a brute force algorithm centered around this notion will assist in attacking 
a password for a previously unseen picture. Zhao et al. describe the BestCover algorithm 
to create a LdGSF sequence dictionary. The program written for this algorithm was not 
released to the public. Hence, we attempted to recreate their algorithm using known pass¬ 
words to derive patterns of data that were used to prioritize guesses, providing the most 
efficient coverage of the password space, i.e., guesses were ordered by popularity of the 
relationship between POI and gesture. Figure 4.3, expresses in pseudocode the BestCover 
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algorithm in a way that aligns with our implementation of the original work of Zhao et al. 

1: function BestCover((si ,..., s* n ), (;fi,..., 7? n )) 

2: for Si in (s“i,..., s* n ) do 

3: for 7Tj in (^i,, j? n ) do 

4: if 7fj e Si then 

5: Si count + + 

6: end if 

7: end for 

8: end for 

9: for Si in (4,..., s* n ) do 

10: if Si count 4- 0 then 

11: S' {5/ : s} count} 

12: end if 

13: end for 

14: order sort S' by max s} count 

15: return order 

16: end function 

Figure 4.3. The Pseudocode of BestCover. Adapted from [3], [4], 

First, the LdGSF sequences were created separately. Each set of attributes collected for the 
LdGSFs was built from known passwords. Since the passwords contain coordinate points, 
if a point fell within an interval of a POI’s location then that attribute and its gesture were 
recorded as an LdGSF. If the coordinate point fell within an intersection of multiple POIs 
then multiple attributes were added in the LdGSF. 

The input to the BestCover algorithm consists of the training data’s LdGSF sequences and 
their corresponding passwords. Lines 2-5 verify the number of passwords that the LdGSF 
sequences produce from the training data, assigning them each a rating. The LdGSF se¬ 
quences not found to produce any of the passwords are not beneficial to the final dictionary 
to produce passwords. In lines 9-11, only the LdGSF sequences with a ranking greater 
than zero are taken into consideration in the dictionary. After zero-rank LdGSF sequences 
are removed, the remaining are ordered by rank in line 14. The highest ranked LdGSF 
sequence is assigned the highest priority because it is viewed as most likely to generate a 
correct password based on its frequency in the test data. The ordered list is then returned 
and used to generate a password dictionary. 

To build the password dictionary, we defined the CreateDictionary algorithm in Fig- 
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ure 4.4 with the main focus being on the sets of attributes in each LdGSF. If even one 
element of the list matches a POI in the given picture, then the LdGSF is beneficial. Oth¬ 
erwise, the entire LdGSF sequence is disregarded. With a valid sequence, a search for all 
POI combinations that match the sequence attributes are found. The list of combinations 
are heuristically ordered by pattern as described in line 5. 

Each PGA password combination is described as positively horizontal if the gestures placed 
in the POI locations appear to be in a left-to-right order, negatively horizontal for a right- 
to-left orientation, positively vertical if the gestures are bottom-to-top, negatively vertical if 
top-to-bottom, and diagonal if they have both a vertical and horizontal pattern. According 
to Zhao et al., user gesture patterns are found to be most common in the following order: 
positively horizontal, diagonal, positively vertical, negatively horizontal, negatively verti¬ 
cal, and the rest follow. These results could not be reproduced in our work therefore, the 
order of password guesses made by Create Diet ionary differed from those in Zhao et al. 
password dictionary. This heuristically ordered list of applicable sequences derived from 
LdGSFs is the final password dictionary. The results collected on the number of password 
guesses may vary based on the assumptions made in designing CreateDictionary. 

1 : function CreateDictionary(o rder, 0 0 
2 : for {s!, s~ 2 , 53 } in order do 

3: for cr\, o" 2 ,(T 3 e Ok do 

4 : if crjes} then 

5: POIlist <— order by Horiz+, Diag, Vert+, Horiz-, Vert- then Other 

6: end if 

7 : end for 

8: end for 

9: for set € POIlist do 

10 : dictionary <— (x,-, yi) e set Vi 

11: end for 

12 : return dictionary 

13: end function 

Figure 4.4. Ordered LdGSFs from Figure 4.3 and an Unseen Picture are 
Used to Brute Force a Password 

Finally, given the algorithms and the data on each picture, we were able to generate pass¬ 
word guesses and keep count of how many guesses were made before each password was 
cracked. These results are analyzed in Chapter 5. 
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CHAPTER 5: 
Analysis 


In this chapter, we analyze the POIs for the pictures in the Arizona-Turk dataset and discuss 
the results of the algorithms described in Chapter 4. 

5.1 Analyzing Points of Interest 

For this research, we analyzed the POIs of the 15 pictures from the Arizona-Turk study, 
shown in Figure 3.1. Due to PII reasons, we did not have access to the pictures in the 
Arizona-Student study, so we could not analyze these. Figure 5.1, shows red dashed rect¬ 
angles on each of the pictures, representing extracted POIs from the images as discussed 
in Section 4.1. The type of POI is labeled above each rectangle. Each POI is identified 
as a face, a body, an eye, an ear, a mouth, a nose, a set of head and shoulders, a clock, 
an airplane, a forehead, or a car. Some POIs were identified only as line, circle, or color 
type. Other POIs were identified as an unknown objects, or as having no semantics. The 
algorithm only used the previously listed POIs when creating passwords. 

The following are the POIs that were identified for each corresponding picture in Fig¬ 
ure 5.1: 

• Figure 5.1(a) is simply a picture of an airplane in the sky, but the POIs identified are 
a nose, a mouth, and another POI with no semantics. 

• Figure 5.1(b) is also an airplane in the sky, yet the POIs identified are two eyes and a 
POI with no semantics. 

• Figure 5.1(c) is a person with the following identified POIs: a body, a face, three eyes, 
three mouths, three noses, 4 circle types, a color type and 2 POIs with no semantics. 

• Figure 5.1(d) is a picture of children playing together with the following POIs iden¬ 
tified: 1 body, 6 mouths, 2 eyes, 6 circle types, 4 color types, and one with no 
semantics. 

• Figure 5.1(e) is the front of a BMW automobile. The POIs recognized are a clock, a 
nose, 5 color types, 3 circle types, and a POI with no semantics. 

• Figure 5.1(f) is a close-up picture of a train. The POIs identified are 2 bodies, 7 circle 
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types, 6 color types, and a POI with no semantics. 

• Figure 5.1(g) is a car with the following POIs identified: a face, a mouth, 2 noses, 2 
eyes, 4 circle types, 7 color types, and 2 POIs with no semantics. 

• Figure 5.1(h) appears to be two tourists standing together. The POIs identified in this 
picture are 2 bodies, 3 mouths, 2 eyes, 5 circle types, 4 color types, a POI with no 
semantics. 

• Figure 5.1(i) is a picture of a man and a woman. The POIs are 6 mouths, 6 eyes, 2 
faces, 5 circle types, and a set of head and shoulders. 

• Figure 5.1(j) is a picture with a group of people. The POIs identified are an eye, 5 
bodies, 7 faces, and 6 mouths. 

• Figure 5.1(k) is another picture of two people with the following identified POIs: a 
body, a nose, 6 mouths, 2 faces, 3 eyes, and 6 circle types. 

• Figure 5.1(1) is a bicycle with the following POIs identified: a body, a face, an eye, 6 
mouths, 6 circle types, and 3 POIs with no semantics. 

• Figure 5.1(m) is also a bicycle with the following POIs identified: a body, a face, an 
eye, 4 mouths, 4 circle types, 2 color types, and 2 with no semantics. 

• Figure 5.1(n) is a picture of a man. The POIs identified are a body, 2 noses, 3 eyes, 4 
mouths, 3 circle types, a color type, a set of head and shoulders, and 4 POIs with no 
semantics. 

• Figure 5.1(o) is a picture of a dog. The POIs found are a nose, 4 eyes, 2 mouths, 3 
circle types, 4 color types, and 2 with no semantics. 

Clearly, many POIs were incorrectly identified, therefore the source of POI extraction ap¬ 
pears not to have been well developed. This led to major consequences when using the 
BestCover algorithm, which is discussed further in Section 5.2. 

Figure 5.2 shows the 15 pictures from the study with their corresponding POI boxes in 
red and associated passwords in blue. Of note, the password coordinate points tend to fall 
within the red POI boxes. Specific shapes were used to guide gestures that were made for 
the passwords, for example heads and wheels were circled, edges had lines associated with 
them, and eyes were tapped. Any password guess with a single gesture outside the scope 
of the picture’s POIs was not cracked. The algorithm made password guesses based only 
on information known about the POIs. We did not make password guesses outside the POI 
boxes shown in red. We did, however, consider circles around POIs, as long as their center 
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(a) 000243 (b) 000316 (c) 001116 (d) 001358 (e) 002057 



(k) 005570 (I) 006412 (m) 006467 (n) 007628 (o) 009899 

Figure 5.1. Identified POIs of the 15 Pictures from the Arizona-Turk Dataset 


point was in a POL For example, Figures 5.2a and 5.2b are pictures of a small airplane 
with a clear sky in the background and with the airplane being the only POI in each picture, 
there were many passwords with gestures made outside of the POI, i.e., in the middle of 
the sky. Table 5.1 represents the percentage of passwords with either one, two, or all three 
of their gestures made within POIs, and indicates the chances of the algorithm cracking a 
password. 

This data showed how often users rely on POIs in creating their passwords. For example, 
in Figure 5.3, by looking only at the passwords for each of these pictures without the 
background pictures themselves, it is clear that the pictures are bicycles. 


5.2 Analyzing BestCover Results 

Implementing the BestCover algorithm (see Section 4.3) on the Arizona-Turk dataset pro¬ 
vided the results shown in Figures 5.4 through 5.16. These graphs only show data from 
passwords that were cracked. The rest of the passwords could not be cracked by the al¬ 
gorithm, therefore, the password guess count is irrelevant.Our results were not comparable 
to Zhao et al. since their experiments used both the Arizona-Turk dataset and the Arizona- 
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Table 5.1. Percentage of Passwords Possible to Guess with Number of Ges¬ 
tures in POIs 


Figure 

% passwords with 
all three gestures 
outside of POIs 

% had exactly two 
gestures outside 
of the POIs 

% had only one 
gesture outside 
of the POIs 

% passwords 
were guessable 
using algorithm 

5.2(a) 

19 

14 

15 

52 

5.2(b) 

20 

19 

15 

46 

5.2(c) 

3 

2 

13 

82 

5.2(d) 

1 

2 

12 

82 

5.2(e) 

7 

13 

16 

86 

5.2(f) 

4 

6 

18 

72 

5.2(g) 

4 

6 

17 

73 

5.2(h) 

6 

8 

21 

65 

5.2(i) 

0 

0 

3 

97 

5.20) 

4 

5 

14 

76 

5.2(k) 

3 

1 

1 

85 

5.2(1) 

0 

0 

6 

94 

5.2(m) 

3 

3 

12 

82 

5.2(n) 

0 

0 

1 

99 

5.2(o) 

5 

6 

19 

70 


Student datasets. 

Figure 5.4 shows the results of Figure 3.1(a). As mentioned in Section 5.1, there are very 
few POIs in this picture, and they were not correctly identified. This made it unlikely 
that the algorithm would crack the password on this type of picture. Less than 30% of 
the passwords were cracked, and the uncracked passwords were those with gestures found 
outside of the POIs. The POIs took up a small area of this picture allowing the algorithm to 
run quickly. A picture with a minimal amount of POIs should not be used as a background 
choice. 

Figure 5.5 shows the results of Figure 3.1(b). Similar to the last picture, there were very 
few POIs in this picture, and yet they were all incorrectly identified. Due to the lack of 
POIs, the algorithm only took a few minutes to run, but only cracked about 30% of the 
passwords due most gestures being made outside of POIs. Since this picture did not have 
many POIs, it is not the best choice for a background. 

Figure 5.6 shows the results of Figure 3.1(c). There were several POIs, most of which were 
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(a) 000243 


(b) 000316 


(c) 001116 


(d) 001358 


(e) 002057 







(f) 002080 


(g) 002840 


(h) 003026 


(i) 003731 


(j) 004054 











(k) 005570 


(I) 006412 


(m) 006467 


(n) 007628 


(o) 009899 


Figure 5.2. Passwords of the 15 Pictures from the Arizona-Turk Dataset 



Figure 5.3. Passwords for Two Pictures of the Arizona-Turk Dataset 


accurately identified, and covered most of the area of the picture, allowing the algorithm 
to crack about 40% of the passwords. Observing the results, we notice that the majority 
of the passwords were cracked within the same range of guesses. This allows us to think 
of improvements for the algorithm. Details for improving the algorithm can be found in 
Section 6.2. Despite the higher password-cracking rate of this picture, this picture is a 
better background choice compared to the previous ones since it has more POIs, but we 
will discuss how some of the other pictures are superior choices. 

Figure 5.7 shows the results of Figure 3.1(d). The algorithm was able to crack over 30% 
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Figure 5.4. CDF Results of Picture 000243.jpg 



Figure 5.5. CDF Results of Picture 000316.jpg 


of the passwords. Observing the results, we notice that about 15% of the passwords were 
cracked within the same range of guesses. Assuming this jump on the graph was made 
from the blowup wheel in the picture, the password guesses could have been made sooner 
with improvements in the algorithm described in Section 6.2. There were unidentified POIs 
in this background picture that were used as guidance for gestures. Since those POIs were 
not identified, the algorithm was unable to crack those passwords. 

Figure 5.8 shows the results of Figure 3.1(e). The algorithm cracked over 30% of the pass¬ 
words. We were unable to determine why so many guesses were made before passwords 
were cracked. It is believed that the overlap caused repeated guesses that should be im- 
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Figure 5.6. CDF Results of Picture 001116.jpg 



Figure 5.7. CDF Results of Picture 001358.jpg 


proved in the algorithm. With the POIs covering only half of the picture and some POIs 
not identified, this was a stronger picture background. 

Figure 5.9 shows the results of Figure 3.1(f). The algorithm cracked about 35% of the 
passwords. About 2/3 of the passwords cracked were within the same range of guesses. 
It is safe to assume these passwords that were cracked were the three wheels on the train. 
This picture is a perfect example to explain how to improve the algorithm to make guesses 
starting with coordinate points in the midpoint of the POI, instead of bottom-left to the 
top-right as the algorithm works. More details can be found in Section 6.2. If the wheels 
were not the main focus of users, this would make a stronger background picture. 
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Figure 5.8. CDF Results of Picture 002057.jpg 



Figure 5.9. CDF Results of Picture 002080.jpg 


Figure 5.10 shows the results of Figure 3.1(g). The algorithm cracked over 30% of the 
passwords. Observing the results, we notice that about 20% of the passwords were quickly 
cracked. With the entire car identified as a POI, these POIs were able to be cracked. 

Figure 5.11 shows the results of Figure 3.1(h). The algorithm cracked about 25% of the 
passwords. Observing the results, we notice that about 10-15% of the passwords cracked 
were from circling the heads. Besides those passwords, it was very difficult to crack other 
passwords with this background since there is so much activity in this picture. This is a 
great example of a secure background picture. 

Figure 5.12 shows the results of Figure 3.1(i). The algorithm cracked over 30% of the 
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Figure 5.10. CDF Results of Picture 002840.jpg 



Figure 5.11. CDF Results of Picture 003026.jpg 


passwords with 25% of them immediately guessed. A close up picture gives less interesting 
POIs of interest, making it easy to guess the passwords. Circling heads, tapping eyes, and 
connecting eyes are the first guesses made. Otherwise, there were not many passwords 
cracked. 

Figure 5.13 shows the results of Figure 3.1(j). The algorithm cracked about 35% of the 
passwords with 25% of them being a combination of circling heads the heads. If users 
were using more of a variety of POIs, then there would be significantly fewer passwords 
cracked. 

Figure 5.14 shows the results of Figure 3.1(m). The algorithm cracked just under 30% of 
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Figure 5.12. CDF Results of Picture 003731.jpg 



Figure 5.13. CDF Results of Picture 004054.jpg 


the passwords. The first 15% of the passwords were using the tires as POIs. Otherwise, the 
other passwords were difficult to crack. This is a decent background picture since there are 
many POIs that can be of interest. 

Figure 5.15 shows the results of Figure 3.1(n). The algorithm cracked about 40% of the 
passwords. The first 15% were immediately identified. They must have been in the same 
class of LdGSFs. With the man’s face being the main focus of passwords chosen by users, 
this is not the best choice of a background picture. 

Figure 5.16 shows the results of Figure 3.1(o). The algorithm cracked about 35% of the 
passwords. About 25% of these passwords were guessed almost simultaneously. Altering 
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Figure 5.14. CDF Results of Picture 006467.jpg 



Figure 5.15. CDF Results of Picture 007628.jpg 


the program to guess these passwords first would be a major improvement. Not many of 
the other passwords were cracked. There was not enough of a variety of POIs in this picture 
for users to vary their passwords, making it a weak background picture. 

Depending on the picture used, perhaps because of the number of POIs in the picture, the 
time taken for the algorithm to break all the passwords varied widely. 

5.3 Algorithm Difficulties and Solutions 

Our results were not directly comparable to Zhao et al.’s results since the testing and train¬ 
ing data used were different, however we were able to create an algorithm that cracks PGA 
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Figure 5.16. CDF Results of Picture 009899.jpg 


passwords. Our algorithm used a significant amount of memory, hard disk space, and CPU 
time to sort and compare the many coordinate points gathered as password guesses, as 
described in Section 4.3. 

Text-based passwords are normally stored using a hash. It is unknown how Microsoft 
stores PGA passwords but our method described in Chapter 2 used a significant amount of 
storage. Python 2 dictionary and list data structures were used to keep track of passwords 
that were cracked and the number of guesses required to crack each password. Suo et al. 
mentioned that memory storage for password guesses is a difficult problem with PGA [2], 
In addition to memory problems, the CPU was not powerful enough on our device to handle 
the amount of work necessary to run the algorithm. 

To address the memory issues and the slow execution on our architecture, we used Amazon 
Web Services (AWS) 3 to run the algorithms. We created an instance of a c4.xlarge Ubuntu 
server with 16 GB of memory and 4 CPUs. Due to cost factors, the time spent using the 
AWS instance was kept to a minimum, roughly $45. The algorithm was run for each of the 
15 pictures, on separate CPUs for efficiency. 

Even with AWS, however, we were unable to find results for Figures 3.1(k) and 3.1(1) 
for which the program failed and never completed. There was no error message, such as 
“MemoryError,” to indicate what caused the failures. Attempts to display an exit status 


2 https://www.python.org/ 

3 https://aws.amazon.com/ 
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in the terminal (i.e., “echo $?”) also failed. The same results were found after running 
the program multiple times for each of those pictures. We assume there was possibly an 
excessive number of passwords generated for these pictures. Perhaps there were far more 
POIs for these than for the other successful pictures. Fortunately, we achieved results for 
the latter pictures. 
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CHAPTER 6: 

Conclusions and Future Work 


In this chapter, we will discuss the accomplishments of this thesis, our recommendations 
to improve the security of PGA, and future work that can be done to continue the research. 

6.1 Conclusions 

Each picture from the Arizona-Turk study was investigated in this thesis for its strength as 
a background picture for PGA. It was found that strong background pictures have a wide 
variety of POIs. More POIs in a picture implies that there are many more gestures a user 
can choose from in creating a password. It is assumed that users will choose from among 
the POIs to assist their choice of password gestures. 

An important benefit of this thesis is the creation of a program that can crack gesture pass¬ 
words. We provided a description on how to crack passwords for PGA. Using data given 
by Zhao et al., we created visual representations demonstrating the POIs and passwords 
of the pictures for the Arizona-Turk study. Visuals were created to show efficiency of the 
program we designed to offer supplementary resources to understand the limits of security 
of PGA. 

Strength requirements for PGA passwords, just as there are for text-based passwords, will 
improve the security of PGA. For example, strength requirements for Windows 8 and Win¬ 
dows 10 might be to increase the number of gestures per password, add new types of 
gestures, and ensure the picture chosen by the user contains numerous POIs dispersed 
across the picture. Using a smaller error distance, as discussed in Section 2.1, will force 
an attacker to make more guesses, however this can cause false negatives when valid users 
attempt to log in. Until such strength requirements are available, we conclude that it would 
be beneficial to use a different means of authentication for the security of government in¬ 
formation. 
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6.2 Future Work 

We have developed a working program that produces sensible guesses to crack PGA pass¬ 
words. Ideally, this program can be improved in the following ways: 

• Most importantly, the algorithm can be enhanced by making fewer guesses. 

• Advancements can also be accomplished by refining memory issues and increasing 
speed. This can be done by using a better POI detection program, and considering 
programming languages other than Python. 

• Since the coordinate points guessed in the algorithm are made in order from the 
bottom-left to the top-right, an improvement might be to randomize the order of 
password guesses in the list of guesses made for each heuristically ordered pattern, 
as described in Section 4.3. 

• Another solution to the same problem may be to begin at the center of each POI, 
which would “hit” the commonly used midpoints of the circle. 

• Furthermore, the algorithm can be designed to construct password guesses outside of 
the POIs in the picture, but at that point, it would be brute-forcing. 

• Finally, it is intended that the program works for unseen pictures. This may be uti¬ 
lized by adding an algorithm that locates POIs and records the coordinate locations 
of the POIs. With this information, the brute-force algorithm in Chapter 4 can guess 
passwords for unseen pictures. 
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